{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to DataFrames\n",
    "**[Bogumił Kamiński](http://bogumilkaminski.pl/about/), Nov 17, 2020**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "using DataFrames"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Manipulating columns of a `DataFrame`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Renaming columns\n",
    "\n",
    "Let's start with a `DataFrame` of `Bool`s that has default column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame(rand(Bool, 3, 4), :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With `rename`, we create new `DataFrame`; here we rename the column `:x1` to `:A`. (`rename` also accepts collections of Pairs.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& A & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m A     \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename(x, :x1 => :A)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With `rename!` we do an in place transformation. \n",
    "\n",
    "This time we've applied a function to every column name (note that the function gets a column names as a string)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1x1</th><th>x2x2</th><th>x3x3</th><th>x4x4</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1x1 & x2x2 & x3x3 & x4x4\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1x1  \u001b[0m\u001b[1m x2x2  \u001b[0m\u001b[1m x3x3  \u001b[0m\u001b[1m x4x4 \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename!(c -> c^2, x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also change the name of a particular column without knowing the original.\n",
    "\n",
    "Here we change the name of the third column, creating a new `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1x1</th><th>x2x2</th><th>third</th><th>x4x4</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1x1 & x2x2 & third & x4x4\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1x1  \u001b[0m\u001b[1m x2x2  \u001b[0m\u001b[1m third \u001b[0m\u001b[1m x4x4 \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename(x, 3 => :third)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we pass a vector of names to `rename!`, we can change the names of all variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m d    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename!(x, [:a, :b, :c, :d])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In all the above examples you could have used strings instead of symbols, e.g."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m d    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename!(x, string.('a':'d'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`rename!` allows for circular renaming of columns, e.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m d    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>d</th><th>b</th><th>c</th><th>a</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& d & b & c & a\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m d     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m a    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename!(x, \"a\"=>\"d\", \"d\"=>\"a\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get an error when we try to provide duplicate names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "ename": "LoadError",
     "evalue": "ArgumentError: Duplicate variable names: :a. Pass makeunique=true to make them unique using a suffix automatically.",
     "output_type": "error",
     "traceback": [
      "ArgumentError: Duplicate variable names: :a. Pass makeunique=true to make them unique using a suffix automatically.",
      "",
      "Stacktrace:",
      " [1] rename!(::DataFrames.Index, ::Array{Symbol,1}; makeunique::Bool) at D:\\AppData\\.julia\\packages\\DataFrames\\X0xNW\\src\\other\\index.jl:51",
      " [2] #rename!#49 at D:\\AppData\\.julia\\packages\\DataFrames\\X0xNW\\src\\abstractdataframe\\abstractdataframe.jl:172 [inlined]",
      " [3] #rename#55 at D:\\AppData\\.julia\\packages\\DataFrames\\X0xNW\\src\\abstractdataframe\\abstractdataframe.jl:296 [inlined]",
      " [4] rename(::DataFrame, ::Array{Symbol,1}) at D:\\AppData\\.julia\\packages\\DataFrames\\X0xNW\\src\\abstractdataframe\\abstractdataframe.jl:296",
      " [5] top-level scope at In[10]:1",
      " [6] include_string(::Function, ::Module, ::String, ::String) at .\\loading.jl:1091"
     ]
    }
   ],
   "source": [
    "rename(x, fill(:a, 4))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " unless we pass `makeunique=true`, which allows us to handle duplicates in passed names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>a_1</th><th>a_2</th><th>a_3</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & a\\_1 & a\\_2 & a\\_3\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m a_1   \u001b[0m\u001b[1m a_2   \u001b[0m\u001b[1m a_3  \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rename(x, fill(:a, 4), makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Reordering columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can reorder the `names(x)` vector as needed, creating a new `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>b</th><th>a</th><th>c</th><th>d</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><th>2</th><td>0</td><td>1</td><td>0</td><td>0</td></tr><tr><th>3</th><td>0</td><td>1</td><td>0</td><td>0</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& b & a & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 1 & 0 & 0 \\\\\n",
       "\t2 & 0 & 1 & 0 & 0 \\\\\n",
       "\t3 & 0 & 1 & 0 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m b     \u001b[0m\u001b[1m a    \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m d     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  true  false  false\n",
       "   2 │ false  true  false  false\n",
       "   3 │ false  true  false  false"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using Random\n",
    "Random.seed!(1234)\n",
    "x[:, shuffle(names(x))]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also `select!` can be used to achieve this in place (or `select` to perform a copy):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>d</th><th>b</th><th>c</th><th>a</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>2</th><td>0</td><td>0</td><td>0</td><td>1</td></tr><tr><th>3</th><td>0</td><td>0</td><td>0</td><td>1</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& d & b & c & a\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 0 & 0 & 0 & 1 \\\\\n",
       "\t2 & 0 & 0 & 0 & 1 \\\\\n",
       "\t3 & 0 & 0 & 0 & 1 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m d     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m a    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ false  false  false  true\n",
       "   2 │ false  false  false  true\n",
       "   3 │ false  false  false  true"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>c</th><th>b</th><th>d</th></tr><tr><th></th><th>Bool</th><th>Bool</th><th>Bool</th><th>Bool</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>1</td><td>0</td><td>0</td><td>0</td></tr><tr><th>2</th><td>1</td><td>0</td><td>0</td><td>0</td></tr><tr><th>3</th><td>1</td><td>0</td><td>0</td><td>0</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & c & b & d\\\\\n",
       "\t\\hline\n",
       "\t& Bool & Bool & Bool & Bool\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 0 & 0 & 0 \\\\\n",
       "\t2 & 1 & 0 & 0 & 0 \\\\\n",
       "\t3 & 1 & 0 & 0 & 0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a    \u001b[0m\u001b[1m c     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m d     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Bool \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\u001b[90m Bool  \u001b[0m\n",
       "─────┼───────────────────────────\n",
       "   1 │ true  false  false  false\n",
       "   2 │ true  false  false  false\n",
       "   3 │ true  false  false  false"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select!(x, 4:-1:1);\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Merging/adding columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame([(i,j) for i in 1:3, j in 1:4], :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With `hcat` we can merge two `DataFrame`s. Also [x y] syntax is supported but only when DataFrames have unique column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x1_1</th><th>x2_1</th><th>x3_1</th><th>x4_1</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 8 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x1\\_1 & x2\\_1 & x3\\_1 & x4\\_1\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×8 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x1_1   \u001b[0m\u001b[1m x2_1   \u001b[0m\u001b[1m x3_1   \u001b[0m\u001b[1m x4_1   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcat(x, x, makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use `hcat` to add a new column; a default name `:x1` will be used for this column, so `makeunique=true` is needed in our case."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x1_1</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Int64</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>1</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>2</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x1\\_1\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & 1 \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & 2 \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x1_1  \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)      1\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)      2\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)      3"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = hcat(x, [1,2,3], makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also prepend a vector with `hcat`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x1_1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Int64</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>1</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>2</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>3</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x1\\_1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & 2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & 3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x1_1   \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │     1  (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │     2  (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │     3  (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcat([1,2,3], x, makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively you could append a vector with the following syntax. This is a bit more verbose but cleaner."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>A</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Int64</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>1</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>2</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & A\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & 1 \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & 2 \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m A     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)      1\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)      2\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)      3"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = [x DataFrame(A=[1,2,3])]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we do the same but add column `:A` to the front."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Int64</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>1</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>2</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>3</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& A & x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & 2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & 3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m A     \u001b[0m\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │     1  (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │     2  (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │     3  (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = [DataFrame(A=[1,2,3]) x]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A column can also be added in the middle. Here a brute-force method is used and a new `DataFrame` is created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  5.517 μs (97 allocations: 9.25 KiB)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>A</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Int64</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>1</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>2</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>3</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & A & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Int64 & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & 1 & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & 2 & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & 3 & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m A     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)      1  (1, 3)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 2)      2  (2, 3)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 2)      3  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "using BenchmarkTools\n",
    "@btime [$x[!, 1:2] DataFrame(A=[1,2,3]) $x[!, 3:4]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could also do this with a specialized in place method `insertcols!`. Let's add `:newcol` to the `DataFrame` `y`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>newcol</th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 6 columns</p><tr><th>1</th><td>1</td><td>1</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>2</td><td>2</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>3</td><td>3</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccccc}\n",
       "\t& A & newcol & x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & 2 & 2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & 3 & 3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×6 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m A     \u001b[0m\u001b[1m newcol \u001b[0m\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64  \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼───────────────────────────────────────────────\n",
       "   1 │     1       1  (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │     2       2  (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │     3       3  (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(y, 2, \"newcol\" => [1,2,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to insert the same column name several times `makeunique=true` is needed as usual."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>A</th><th>newcol_1</th><th>newcol</th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 7 columns</p><tr><th>1</th><td>1</td><td>1</td><td>1</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>2</td><td>2</td><td>2</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>3</td><td>3</td><td>3</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccccc}\n",
       "\t& A & newcol\\_1 & newcol & x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & 2 & 2 & 2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & 3 & 3 & 3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×7 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m A     \u001b[0m\u001b[1m newcol_1 \u001b[0m\u001b[1m newcol \u001b[0m\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64    \u001b[0m\u001b[90m Int64  \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼─────────────────────────────────────────────────────────\n",
       "   1 │     1         1       1  (1, 1)  (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │     2         2       2  (2, 1)  (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │     3         3       3  (3, 1)  (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(y, 2, :newcol => [1,2,3], makeunique=true)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see how much faster it is to insert a column with `insertcols!` than with `hcat` using `@btime` (note that we use here a `Pair` notation as an example)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  1.260 μs (25 allocations: 2.84 KiB)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>A</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Int64</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>1</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>2</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>3</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & A & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Int64 & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & 1 & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & 2 & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & 3 & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m A     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)      1  (1, 3)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 2)      2  (2, 3)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 2)      3  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@btime insertcols!(copy($x), 3, :A => [1,2,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's use `insertcols!` to append a column in place (note that we dropped the index at which we insert the column)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>A</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Int64</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>1</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>2</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & A\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & 1 \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & 2 \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m A     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)      1\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)      2\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)      3"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(x, :A => [1,2,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and to in place prepend a column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>B</th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>A</th></tr><tr><th></th><th>Int64</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Int64</th></tr></thead><tbody><p>3 rows × 6 columns</p><tr><th>1</th><td>1</td><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>1</td></tr><tr><th>2</th><td>2</td><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>2</td></tr><tr><th>3</th><td>3</td><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccccc}\n",
       "\t& B & x1 & x2 & x3 & x4 & A\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Tuple… & Tuple… & Tuple… & Tuple… & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & 1 \\\\\n",
       "\t2 & 2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & 2 \\\\\n",
       "\t3 & 3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×6 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m B     \u001b[0m\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m A     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼──────────────────────────────────────────────\n",
       "   1 │     1  (1, 1)  (1, 2)  (1, 3)  (1, 4)      1\n",
       "   2 │     2  (2, 1)  (2, 2)  (2, 3)  (2, 4)      2\n",
       "   3 │     3  (3, 1)  (3, 2)  (3, 3)  (3, 4)      3"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(x, 1, :B => [1,2,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `insertcols!` can be used to insert several columns to a data frame at once and that it performs broadcasting if needed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th></tr><tr><th></th><th>Int64</th></tr></thead><tbody><p>3 rows × 1 columns</p><tr><th>1</th><td>1</td></tr><tr><th>2</th><td>2</td></tr><tr><th>3</th><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& a\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\t3 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2\n",
       "   3 │     3"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(a = [1, 2, 3])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th><th>d</th></tr><tr><th></th><th>Int64</th><th>String</th><th>Char</th><th>Array…</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>1</td><td>x</td><td>a</td><td>[1, 2, 3]</td></tr><tr><th>2</th><td>2</td><td>x</td><td>b</td><td>[1, 2, 3]</td></tr><tr><th>3</th><td>3</td><td>x</td><td>c</td><td>[1, 2, 3]</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& a & b & c & d\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & String & Char & Array…\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & x & a & [1, 2, 3] \\\\\n",
       "\t2 & 2 & x & b & [1, 2, 3] \\\\\n",
       "\t3 & 3 & x & c & [1, 2, 3] \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b      \u001b[0m\u001b[1m c    \u001b[0m\u001b[1m d         \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m String \u001b[0m\u001b[90m Char \u001b[0m\u001b[90m Array…    \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │     1  x       a     [1, 2, 3]\n",
       "   2 │     2  x       b     [1, 2, 3]\n",
       "   3 │     3  x       c     [1, 2, 3]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(df, :b => \"x\", :c => 'a':'c', :d => Ref([1,2,3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Interestingly we can emulate `hcat` mutating the data frame in-place using `insertcols!`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th></tr><tr><th></th><th>Int64</th></tr></thead><tbody><p>2 rows × 1 columns</p><tr><th>1</th><td>1</td></tr><tr><th>2</th><td>2</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& a\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 = DataFrame(a=[1,2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>b</th><th>c</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>2 rows × 2 columns</p><tr><th>1</th><td>2</td><td>3</td></tr><tr><th>2</th><td>3</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& b & c\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 2 & 3 \\\\\n",
       "\t2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼──────────────\n",
       "   1 │     2      3\n",
       "   2 │     3      4"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = DataFrame(b=[2,3], c=[3,4])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>2 rows × 3 columns</p><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr><tr><th>2</th><td>2</td><td>3</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& a & b & c\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\t2 & 2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      2      3\n",
       "   2 │     2      3      4"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcat(df1, df2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th></tr><tr><th></th><th>Int64</th></tr></thead><tbody><p>2 rows × 1 columns</p><tr><th>1</th><td>1</td></tr><tr><th>2</th><td>2</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& a\\\\\n",
       "\t\\hline\n",
       "\t& Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 \\\\\n",
       "\t2 & 2 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────\n",
       "   1 │     1\n",
       "   2 │     2"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 # df1 is not touched"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>2 rows × 3 columns</p><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr><tr><th>2</th><td>2</td><td>3</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& a & b & c\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\t2 & 2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      2      3\n",
       "   2 │     2      3      4"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "insertcols!(df1, pairs(eachcol(df2))...)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>a</th><th>b</th><th>c</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>2 rows × 3 columns</p><tr><th>1</th><td>1</td><td>2</td><td>3</td></tr><tr><th>2</th><td>2</td><td>3</td><td>4</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& a & b & c\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 2 & 3 \\\\\n",
       "\t2 & 2 & 3 & 4 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m2×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m a     \u001b[0m\u001b[1m b     \u001b[0m\u001b[1m c     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      2      3\n",
       "   2 │     2      3      4"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df1 # now we have changed df1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Subsetting/removing columns\n",
    "\n",
    "Let's create a new `DataFrame` `x` and show a few ways to create DataFrames with a subset of `x`'s columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame([(i,j) for i in 1:3, j in 1:5], :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First we could do this by index:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:, [1,2,4,5]] # use ! instead of : for non-copying operation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or by column name:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x4</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 4) \\\\\n",
       "\t2 & (2, 1) & (2, 4) \\\\\n",
       "\t3 & (3, 1) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 4)\n",
       "   2 │ (2, 1)  (2, 4)\n",
       "   3 │ (3, 1)  (3, 4)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:, [:x1, :x4]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also choose to keep or exclude columns by `Bool` (we need a vector whose length is the number of columns in the original `DataFrame`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x3 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 5)"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:, [true, false, true, false, true]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a single column `DataFrame`,"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th></tr><tr><th></th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 1 columns</p><tr><th>1</th><td>(1, 1)</td></tr><tr><th>2</th><td>(2, 1)</td></tr><tr><th>3</th><td>(3, 1)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x1\\\\\n",
       "\t\\hline\n",
       "\t& Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) \\\\\n",
       "\t2 & (2, 1) \\\\\n",
       "\t3 & (3, 1) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────\n",
       "   1 │ (1, 1)\n",
       "   2 │ (2, 1)\n",
       "   3 │ (3, 1)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[:, [:x1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and here we access the vector contained in column `:x1`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Tuple{Int64,Int64},1}:\n",
       " (1, 1)\n",
       " (2, 1)\n",
       " (3, 1)"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, :x1] # use : instead of ! to copy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Tuple{Int64,Int64},1}:\n",
       " (1, 1)\n",
       " (2, 1)\n",
       " (3, 1)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.x1 # the same"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could grab the same vector by column number"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Tuple{Int64,Int64},1}:\n",
       " (1, 1)\n",
       " (2, 1)\n",
       " (3, 1)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that getting a single column returns it without copying while creating a new `DataFrame` performs a copy of the column"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "false"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, 1] === x[!, [1]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "you can also use `Regex`, `All`, `Between` and `Not` from InvertedIndies.jl for column selection:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x2\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) \\\\\n",
       "\t2 & (2, 1) & (2, 2) \\\\\n",
       "\t3 & (3, 1) & (3, 2) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 2)\n",
       "   2 │ (2, 1)  (2, 2)\n",
       "   3 │ (3, 1)  (3, 2)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, r\"[12]\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, Not(1)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 3) & (1, 4) \\\\\n",
       "\t2 & (2, 2) & (2, 3) & (2, 4) \\\\\n",
       "\t3 & (3, 2) & (3, 3) & (3, 4) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 2)  (1, 3)  (1, 4)\n",
       "   2 │ (2, 2)  (2, 3)  (2, 4)\n",
       "   3 │ (3, 2)  (3, 3)  (3, 4)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, Between(:x2, :x4)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, Cols(:x1, Between(:x3, :x5))]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(x, :x1, Between(:x3, :x5), copycols=false) # the same as above"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "you can use `select` and `select!` functions to select a subset of columns from a data frame. `select` creates a new data frame and `select!` operates in place"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = copy(x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x2\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) \\\\\n",
       "\t2 & (2, 1) & (2, 2) \\\\\n",
       "\t3 & (3, 1) & (3, 2) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 2)\n",
       "   2 │ (2, 1)  (2, 2)\n",
       "   3 │ (3, 1)  (3, 2)"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = select(df, [1, 2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, Not([1, 2]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "by default `select` copies columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "false"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2[!, 1] === df[!, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "this can be avoided by using `copycols=false` keyword argument"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x2\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) \\\\\n",
       "\t2 & (2, 1) & (2, 2) \\\\\n",
       "\t3 & (3, 1) & (3, 2) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 2)\n",
       "   2 │ (2, 1)  (2, 2)\n",
       "   3 │ (3, 1)  (3, 2)"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2 = select(df, [1, 2], copycols=false)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df2[!, 1] === df[!, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "using `select!` will modify the source data frame"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & x2\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) \\\\\n",
       "\t2 & (2, 1) & (2, 2) \\\\\n",
       "\t3 & (3, 1) & (3, 2) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────\n",
       "   1 │ (1, 1)  (1, 2)\n",
       "   2 │ (2, 1)  (2, 2)\n",
       "   3 │ (3, 1)  (3, 2)"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select!(df, [1,2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df == df2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we create a copy of `x` and delete the 3rd column from the copy with `select!` and `Not`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "z = copy(x)\n",
    "select!(z, Not(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "alternatively we can achieve the same by using the `select` function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(x, Not(3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`x` stays unchanged"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Views"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note, that you can also create a view of a `DataFrame` when we want a subset of its columns:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  872.340 ns (17 allocations: 1.98 KiB)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x3 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 5)"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@btime x[:, [1,3,5]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  195.330 ns (3 allocations: 320 bytes)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x3</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 3)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 3)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 3)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x3 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 3) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 3) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 3) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 SubDataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────\n",
       "   1 │ (1, 1)  (1, 3)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 3)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 3)  (3, 5)"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "@btime @view x[:, [1,3,5]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(now creation of the `view` is slow, but in the coming releases of the DataFrames.jl package it will become significantly faster)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Modify column by name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame([(i,j) for i in 1:3, j in 1:5], :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With the following syntax, the existing column is modified without performing any copying (this is discouraged as it creates column alias)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 2)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 2) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 2) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 2)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 2)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 2)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, :x1] = x[!, :x2]\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "this syntax is safer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Tuple{Int64,Int64},1}:\n",
       " (1, 2)\n",
       " (2, 2)\n",
       " (3, 2)"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, :x1] = x[:, :x2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can also use the following syntax to add a new column at the end of a `DataFrame`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th><th>A</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Int64</th></tr></thead><tbody><p>3 rows × 6 columns</p><tr><th>1</th><td>(1, 2)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td><td>1</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td><td>2</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5 & A\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 2) & (1, 3) & (1, 4) & (1, 5) & 1 \\\\\n",
       "\t2 & (2, 2) & (2, 2) & (2, 3) & (2, 4) & (2, 5) & 2 \\\\\n",
       "\t3 & (3, 2) & (3, 2) & (3, 3) & (3, 4) & (3, 5) & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×6 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\u001b[1m A     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────────────────────────────────────────────\n",
       "   1 │ (1, 2)  (1, 2)  (1, 3)  (1, 4)  (1, 5)      1\n",
       "   2 │ (2, 2)  (2, 2)  (2, 3)  (2, 4)  (2, 5)      2\n",
       "   3 │ (3, 2)  (3, 2)  (3, 3)  (3, 4)  (3, 5)      3"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x[!, :A] = [1,2,3]\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A new column name will be added to our `DataFrame` with the following syntax as well:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th><th>A</th><th>B</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 7 columns</p><tr><th>1</th><td>(1, 2)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td><td>1</td><td>11</td></tr><tr><th>2</th><td>(2, 2)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td><td>2</td><td>12</td></tr><tr><th>3</th><td>(3, 2)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td><td>3</td><td>13</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5 & A & B\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple… & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 2) & (1, 2) & (1, 3) & (1, 4) & (1, 5) & 1 & 11 \\\\\n",
       "\t2 & (2, 2) & (2, 2) & (2, 3) & (2, 4) & (2, 5) & 2 & 12 \\\\\n",
       "\t3 & (3, 2) & (3, 2) & (3, 3) & (3, 4) & (3, 5) & 3 & 13 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×7 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\u001b[1m A     \u001b[0m\u001b[1m B     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼──────────────────────────────────────────────────────\n",
       "   1 │ (1, 2)  (1, 2)  (1, 3)  (1, 4)  (1, 5)      1     11\n",
       "   2 │ (2, 2)  (2, 2)  (2, 3)  (2, 4)  (2, 5)      2     12\n",
       "   3 │ (3, 2)  (3, 2)  (3, 3)  (3, 4)  (3, 5)      3     13"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.B = 11:13\n",
    "x"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Find column name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>x5</th></tr><tr><th></th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th><th>Tuple…</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>(1, 1)</td><td>(1, 2)</td><td>(1, 3)</td><td>(1, 4)</td><td>(1, 5)</td></tr><tr><th>2</th><td>(2, 1)</td><td>(2, 2)</td><td>(2, 3)</td><td>(2, 4)</td><td>(2, 5)</td></tr><tr><th>3</th><td>(3, 1)</td><td>(3, 2)</td><td>(3, 3)</td><td>(3, 4)</td><td>(3, 5)</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & x5\\\\\n",
       "\t\\hline\n",
       "\t& Tuple… & Tuple… & Tuple… & Tuple… & Tuple…\\\\\n",
       "\t\\hline\n",
       "\t1 & (1, 1) & (1, 2) & (1, 3) & (1, 4) & (1, 5) \\\\\n",
       "\t2 & (2, 1) & (2, 2) & (2, 3) & (2, 4) & (2, 5) \\\\\n",
       "\t3 & (3, 1) & (3, 2) & (3, 3) & (3, 4) & (3, 5) \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1     \u001b[0m\u001b[1m x2     \u001b[0m\u001b[1m x3     \u001b[0m\u001b[1m x4     \u001b[0m\u001b[1m x5     \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\u001b[90m Tuple… \u001b[0m\n",
       "─────┼────────────────────────────────────────\n",
       "   1 │ (1, 1)  (1, 2)  (1, 3)  (1, 4)  (1, 5)\n",
       "   2 │ (2, 1)  (2, 2)  (2, 3)  (2, 4)  (2, 5)\n",
       "   3 │ (3, 1)  (3, 2)  (3, 3)  (3, 4)  (3, 5)"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x = DataFrame([(i,j) for i in 1:3, j in 1:5], :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can check if a column with a given name exists via"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "true"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hasproperty(x, :x1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and determine its index via"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "columnindex(x, :x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Advanced ways of column selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "these are most useful for non-standard column names (e.g. containing spaces)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>column 2</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>1</td><td>4</td></tr><tr><th>2</th><td>2</td><td>5</td></tr><tr><th>3</th><td>3</td><td>6</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & column 2\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 \\\\\n",
       "\t2 & 2 & 5 \\\\\n",
       "\t3 & 3 & 6 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m column 2 \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64    \u001b[0m\n",
       "─────┼─────────────────\n",
       "   1 │     1         4\n",
       "   2 │     2         5\n",
       "   3 │     3         6"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame()\n",
    "df.x1 = 1:3\n",
    "df[!, \"column 2\"] = 4:6\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Int64,1}:\n",
       " 4\n",
       " 5\n",
       " 6"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.\"column 2\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Int64,1}:\n",
       " 4\n",
       " 5\n",
       " 6"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[:, \"column 2\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or you can interpolate column name using `:()` syntax"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x1\n",
      "[1, 2, 3]\n",
      "\n",
      "column 2\n",
      "[4, 5, 6]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for n in names(df)\n",
    "    println(n, \"\\n\", df.:($n), \"\\n\")\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Working on a collection of columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When using `eachcol` of a data frame the resulting object retains reference to its parent and e.g. can be queried with `getproperty`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>1</td><td>4</td><td>7</td><td>10</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td><td>11</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td><td>12</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 & 10 \\\\\n",
       "\t2 & 2 & 5 & 8 & 11 \\\\\n",
       "\t3 & 3 & 6 & 9 & 12 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1      4      7     10\n",
       "   2 │     2      5      8     11\n",
       "   3 │     3      6      9     12"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(reshape(1:12, 3, 4), :auto)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<p>3×4 DataFrameColumns</p><table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><tr><th>1</th><td>1</td><td>4</td><td>7</td><td>10</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td><td>11</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td><td>12</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 & 10 \\\\\n",
       "\t2 & 2 & 5 & 8 & 11 \\\\\n",
       "\t3 & 3 & 6 & 9 & 12 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrameColumns\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1      4      7     10\n",
       "   2 │     2      5      8     11\n",
       "   3 │     3      6      9     12"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ec_df = eachcol(df)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Int64,1}:\n",
       " 1\n",
       " 2\n",
       " 3"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ec_df[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3-element Array{Int64,1}:\n",
       " 1\n",
       " 2\n",
       " 3"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ec_df.x1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Transforming columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will get to this subject later in 10_transforms.ipynb notebook, but here let us just note that `select`, `select!`, `transform`, `transform!` and `combine` functions allow to generate new columns based on the old columns of a data frame.\n",
    "\n",
    "The general rules are the following:\n",
    "* `select` and `transform` always return the number of rows equal to the source data frame, while `combine` returns any number of rows (`combine` is allowed to *combine* rows of the source data frame)\n",
    "* `transform` retains columns from the old data frame\n",
    "* `select!` and `transform!` are in-place versions of `select` and `transform`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 4 columns</p><tr><th>1</th><td>1</td><td>4</td><td>7</td><td>10</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td><td>11</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td><td>12</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cccc}\n",
       "\t& x1 & x2 & x3 & x4\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 & 10 \\\\\n",
       "\t2 & 2 & 5 & 8 & 11 \\\\\n",
       "\t3 & 3 & 6 & 9 & 12 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×4 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1      4      7     10\n",
       "   2 │     2      5      8     11\n",
       "   3 │     3      6      9     12"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = DataFrame(reshape(1:12, 3, 4), :auto)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we add a new column `:res` that is a sum of columns `:x1` and `:x2`. A general syntax of transformations of this kind is:\n",
    "\n",
    "```\n",
    "source_columns => function_to_apply => target_column_name\n",
    "```\n",
    "then `function_to_apply` gets columns selected by `source_columns` as positional arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x3</th><th>x4</th><th>res</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 5 columns</p><tr><th>1</th><td>1</td><td>4</td><td>7</td><td>10</td><td>5</td></tr><tr><th>2</th><td>2</td><td>5</td><td>8</td><td>11</td><td>7</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td><td>12</td><td>9</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccccc}\n",
       "\t& x1 & x2 & x3 & x4 & res\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 7 & 10 & 5 \\\\\n",
       "\t2 & 2 & 5 & 8 & 11 & 7 \\\\\n",
       "\t3 & 3 & 6 & 9 & 12 & 9 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×5 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x3    \u001b[0m\u001b[1m x4    \u001b[0m\u001b[1m res   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼───────────────────────────────────\n",
       "   1 │     1      4      7     10      5\n",
       "   2 │     2      5      8     11      7\n",
       "   3 │     3      6      9     12      9"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transform(df, [:x1, :x2] => (+) => :res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One can omit passing `target_column_name` in which case it is automatically generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [],
   "source": [
    "using Statistics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1_x2_cor</th></tr><tr><th></th><th>Float64</th></tr></thead><tbody><p>1 rows × 1 columns</p><tr><th>1</th><td>1.0</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x1\\_x2\\_cor\\\\\n",
       "\t\\hline\n",
       "\t& Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1.0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m1×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1_x2_cor \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Float64   \u001b[0m\n",
       "─────┼───────────\n",
       "   1 │       1.0"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "combine(df, [:x1, :x2] => cor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that `combine` allowed the number of columns in the resulting data frame to be changed. If we used `select` instead it would automatically broadcast the return value to match the number of rouws of the source:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1_x2_cor</th></tr><tr><th></th><th>Float64</th></tr></thead><tbody><p>3 rows × 1 columns</p><tr><th>1</th><td>1.0</td></tr><tr><th>2</th><td>1.0</td></tr><tr><th>3</th><td>1.0</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|c}\n",
       "\t& x1\\_x2\\_cor\\\\\n",
       "\t\\hline\n",
       "\t& Float64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1.0 \\\\\n",
       "\t2 & 1.0 \\\\\n",
       "\t3 & 1.0 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×1 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1_x2_cor \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Float64   \u001b[0m\n",
       "─────┼───────────\n",
       "   1 │       1.0\n",
       "   2 │       1.0\n",
       "   3 │       1.0"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, [:x1, :x2] => cor)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to apply some function on each row of the source wrap it in `ByRow`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x1_x2_string</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>String</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>4</td><td>14</td></tr><tr><th>2</th><td>2</td><td>5</td><td>25</td></tr><tr><th>3</th><td>3</td><td>6</td><td>36</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x2 & x1\\_x2\\_string\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & String\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 14 \\\\\n",
       "\t2 & 2 & 5 & 25 \\\\\n",
       "\t3 & 3 & 6 & 36 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x1_x2_string \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m String       \u001b[0m\n",
       "─────┼────────────────────────────\n",
       "   1 │     1      4  14\n",
       "   2 │     2      5  25\n",
       "   3 │     3      6  36"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, :x2, [:x1, :x2] => ByRow(string))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Also if you want columns to be passed as a `NamedTuple` to a funcion (instead of being positional arguments) wrap them in `AsTable`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x2</th><th>x1_x2_function</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>4</td><td>5</td></tr><tr><th>2</th><td>2</td><td>5</td><td>7</td></tr><tr><th>3</th><td>3</td><td>6</td><td>9</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x2 & x1\\_x2\\_function\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 4 & 5 \\\\\n",
       "\t2 & 2 & 5 & 7 \\\\\n",
       "\t3 & 3 & 6 & 9 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x2    \u001b[0m\u001b[1m x1_x2_function \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64          \u001b[0m\n",
       "─────┼──────────────────────────────\n",
       "   1 │     1      4               5\n",
       "   2 │     2      5               7\n",
       "   3 │     3      6               9"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, :x2, AsTable([:x1, :x2]) => x -> x.x1 + x.x2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For simplicity (as this functionality is often needed) there is a special treatement of `nrow` function that can be just passed as a transformation (without specifying of column selector):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>number_of_rows</th></tr><tr><th></th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 2 columns</p><tr><th>1</th><td>1</td><td>3</td></tr><tr><th>2</th><td>2</td><td>3</td></tr><tr><th>3</th><td>3</td><td>3</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|cc}\n",
       "\t& x1 & number\\_of\\_rows\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 3 \\\\\n",
       "\t2 & 2 & 3 \\\\\n",
       "\t3 & 3 & 3 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×2 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m number_of_rows \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64          \u001b[0m\n",
       "─────┼───────────────────────\n",
       "   1 │     1               3\n",
       "   2 │     2               3\n",
       "   3 │     3               3"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, nrow => \"number_of_rows\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(note that in `select` the number of rows is automatically broadcasted to match the number of rows of the source data frame)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally you can conveninently create multiple columns with one function, e.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x1²</th><th>x1³</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>1</td><td>1</td></tr><tr><th>2</th><td>2</td><td>4</td><td>8</td></tr><tr><th>3</th><td>3</td><td>9</td><td>27</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x1² & x1³\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 1 \\\\\n",
       "\t2 & 2 & 4 & 8 \\\\\n",
       "\t3 & 3 & 9 & 27 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x1²   \u001b[0m\u001b[1m x1³   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      1      1\n",
       "   2 │     2      4      8\n",
       "   3 │     3      9     27"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, :x1 => ByRow(x -> [x ^ 2, x ^ 3]) => [\"x1²\", \"x1³\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or e.g. (this produces the same result)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class=\"data-frame\"><thead><tr><th></th><th>x1</th><th>x1²</th><th>x1³</th></tr><tr><th></th><th>Int64</th><th>Int64</th><th>Int64</th></tr></thead><tbody><p>3 rows × 3 columns</p><tr><th>1</th><td>1</td><td>1</td><td>1</td></tr><tr><th>2</th><td>2</td><td>4</td><td>8</td></tr><tr><th>3</th><td>3</td><td>9</td><td>27</td></tr></tbody></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{r|ccc}\n",
       "\t& x1 & x1² & x1³\\\\\n",
       "\t\\hline\n",
       "\t& Int64 & Int64 & Int64\\\\\n",
       "\t\\hline\n",
       "\t1 & 1 & 1 & 1 \\\\\n",
       "\t2 & 2 & 4 & 8 \\\\\n",
       "\t3 & 3 & 9 & 27 \\\\\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "\u001b[1m3×3 DataFrame\u001b[0m\n",
       "\u001b[1m Row \u001b[0m│\u001b[1m x1    \u001b[0m\u001b[1m x1²   \u001b[0m\u001b[1m x1³   \u001b[0m\n",
       "\u001b[1m     \u001b[0m│\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\u001b[90m Int64 \u001b[0m\n",
       "─────┼─────────────────────\n",
       "   1 │     1      1      1\n",
       "   2 │     2      4      8\n",
       "   3 │     3      9     27"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "select(df, :x1, :x1 => (x -> DataFrame(\"x1²\" => x .^ 2, \"x1³\" => x .^ 3)) => AsTable)"
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "kernelspec": {
   "display_name": "Julia 1.5.3",
   "language": "julia",
   "name": "julia-1.5"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.5.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
